Control device for detection

ABSTRACT

A method for detecting a display is provided, the method including receiving an image signal from an image detector; detecting, by an image processor, that the received image signal includes an image of at least a portion of a display; determining, by the image processor, an array of pixel coordinates of the received image signal that define at least a part of an outline of the display; calculating, based on the array of pixel coordinates and at least one centre pixel of the received image signal, an orientation of the image detector in relation to the display. An apparatus for detecting a display is also provided. The apparatus is configured to carry out the method and includes an image detector and an image processor configured to be in wired or wireless communication with a display.

This invention relates to improvements in or relating to a method fordetecting a display and an apparatus for carrying out the method, theapparatus comprising a control device and, in particular, the inventionrelates to a method for detecting a display device using a line of sightcontrol device as a controller and an apparatus for carrying out themethod comprising a line of sight control device to detect its alignmentrelative to a display device. In addition, the present invention relatesto a line of sight control device to detect the display screen.

It is commonly known that some controllers can be used to interact withobjects on a display screen. A user often uses a controller for manypurposes. For example; for providing an indicator on a screen where theuser is pointing; for detecting objects on a screen for entertainmentpurposes such as playing video games; and/or for using a controller fortraining and educational purposes. Unfortunately, many controllers usedfor detection can be inaccurate and do not truly reflect the position ofwhere the user using the controller points to on a display screen. Forgaming purposes, the controller can be used as a hand-held controllerdevice or a gaming accessory such as a gun-style accessory. Many usersin the past have enjoyed using lightgun technology for playing shootinggames.

However, classic lightgun technology does not work on modem displayssuch as LCD/LED televisions or LCD projectors. Classic light guns usethe fast line by line refresh of the cathode ray tube (CRT) that hasprecise timing. Modem LCD/LED televisions or LCD projectors process theimage as a complete image and then refresh the display as a completeframe usually at a slower rate than a CRT. As a result, users who playclassic lightgun computer games are not able to play with a classiclightgun on a modem television.

The current alternatives are using a mouse, keyboard and joystick tomove the cursor or for a gun style solution putting some extra hardwaresuch as infrared LEDs next to the television that can be detected at thefront of the remote/gun to deduce approximately where the remote/gun ispointing on the display screen. This does not provide true line of sightresponse and provides a less accurate way to calculate where the controldevice is pointing at the display, unless the system goes through acalibration routine.

However, a calibration routine is often a time consuming process becausea user must go through a calibration routine every time the user changestheir position, angle and/or changes their equipment.

It is against this background that the invention has arisen.

According to an aspect of the present invention, there is provided amethod for detecting a display, the method comprising the steps of:

receiving an image signal from an image detection means;

detecting, by an image processor, that the received image signalcomprises an image of at least a portion of a display;

determining, by the image processor, an array of pixel coordinates ofthe received image signal that define at least a part of an outline ofthe display;

calculating, based on the array of pixel coordinates and at least onecentre pixel of the received image signal, an orientation of the imagedetection means in relation to the display.

According to another aspect of the invention, there is provided anapparatus configured to detect a display according to the method of thepresent invention, the apparatus comprising:

-   -   an image detection means; and    -   an image processor configured to be in wired or wireless        communication with a display.

In some embodiments, the apparatus for detecting a display comprises:

-   -   a control device,    -   an artefact being applied to a display content within a display        area of the display, wherein the applied artefact may be        configured to aid the detection of the display,    -   an image detection means configured to capture an image        comprising the display; and    -   an image processor for receiving a first signal from the image        detection means, the image processor is configured to process        the captured image to determine an array of image pixel        coordinates of the display which is compared to an image pixel        coordinates of the centre of the captured image, and    -   wherein the image processor further determines a pointing        alignment of the control device in relation to the display on        the basis of the determined image pixel coordinates to detect        the display.

According to another aspect of the present invention, there is provideda computer-readable medium comprising instructions which, when executedby a processor, cause the processor to carry out the methods describedherein.

In some embodiments, the applied artefact can be a background,foreground, a border or a partial border adapted to partially surroundor completely surround the display content within the display area. Theartefact may be partially transparent. The display area may be a displayscreen of the display.

In some embodiments, the background, foreground, partial border or theborder may be applied to/around at least one edge or corner of thedisplay area. In some embodiments, the applied artefact may be arectangular or square border around the inner edge of the displaycontent to assist the detection of the display area.

In some embodiments the artefact may be based on an algorithm that takesinto account pixel attributes such as, but not limited to, pixellocation, colour and brightness. The algorithm will then appropriatelychange pixel values such as, but not limited to, colour and brightnesswith the purpose of making the image recognition process easier and moreaccurate.

The apparatus may further comprise the application of additionalartefacts around at least one edge or corner of the display content.

According to another aspect of the present invention there is provided,an apparatus for detecting a display device, the apparatus comprising:

-   -   a control device,    -   an artefact being applied to a display content within a display        area of the display, wherein the applied artefact may be        configured to aid the detection of the display,    -   an image detection means configured to capture an image        comprising the display; and    -   an image processor for receiving a first signal from the image        detection means, the image processor is configured to process        the captured image to determine an array of image pixel        coordinates of the display which is compared to an image pixel        coordinates of the centre of the captured image,    -   wherein the image processor further determines a pointing        alignment of the control device in relation to the display on        the basis of the determined image pixel coordinates to detect        the display.

The apparatus of the present invention may be configured to detect,capture, collect multiple signals, process the captured image andanalyse the captured image in real time to determine where the pointingalignment, position, angle and distance of the control device is inrelation to the display. As the control device can be directly used fordetecting the display, even if a user handling the control devicechanges their operating alignment/position, distance from the display orangle to the display the control device does not require calibration andrecalibration. Thus, the apparatus of the present invention does notrequire a calibration routine when in use, which can significantlyimprove the control device for detecting, processing and analysing theimage, and a user experience handling the control device. In anotheradvantage, the apparatus of present invention does not require anyexternal hardware for detecting the relative location of the displaydevice, such as an infra-red LED array next to/or around the displaydevice.

In addition, the present invention provides an advantage in that it canbe used detect modem technology display devices such as LCD or LED,television screens, computer monitors, modem projector displays, mobilephone devices, tablets, cinematic screens or gaming devices.

It is also an advantage that the apparatus of the present invention maybe able to provide a more accurate way to detect an object in thecaptured image, such as the display device.

The control device may be a line of sight control device. The controldevice may be a hand-held device such as a controller. The controllermay be mobile and be operable by a user's left hand, right hand or both.The line of sight controller may comprise a suitable configuration or itmay comprise a strap for the user's hand to grip the controller tooperate the line of sight controller freely for example, the user canrotate the controller, move the controller from side to side and/or upor down.

In some alternative embodiments, the image processor may furtherdetermine the pointing alignment, position, angle and distance of thecontrol device in relation to the display based on the image pixelcoordinates.

Processing the captured image using the image processor, which may be alocal image processor, reduces the transmission time for sending theprocessed data, and it may reduce the amount of hardware on anelectronic device being controlled. The image processor may beconfigured to process multiple signals received from the imagedetections means in real time to determine the array of image pixelcoordinates of the display. In some embodiments, processing of thecaptured image can be performed by the image processor on board thecontrol device or the raw image/video input can be fed to an electronicdevice and be processed by the electronic device, such as a computer ora computer-implemented device. This reduces the amount of hardwarerequired and the complexity of the hardware required in the controldevice.

In some embodiments, the apparatus may further comprise an electronicdevice which may be configured to provide an indicator to display thepointing alignment, position, angle or distance of the control device onthe display. The electronic device may receive a second signal from theimage processor and may send an output signal to the display device todisplay the pointing alignment/position of the control device on thedisplay.

The indicator may be a computer mouse cursor or a targeting crosshairused to display the pointing alignment/position of the controller on thedisplay such as a TV screen, a laptop screen, a computer screen, aprojector screen, a cinematic screen, mobile phone screen, tablets, oron any other electronic screens.

This can be advantageous because the indicator can be used to accuratelyshow where a user handling the control device is pointing to on thescreen. The indicator may be programmable to enable it to automaticallyand accurately show where a user handling control device is pointing toon the screen. In some alternative embodiments, the control device maycomprise an indicator for displaying the pointing alignment/position onthe display device.

In some embodiments, an indicator may not be required. This may give adifferent user experience and the user may rely on the line of sightaccuracy to know where they are pointing on the display.

In some embodiments, the indicator may be a tracking indicator such as atracking cursor which can continually track and display the pointingalignment or position of the control device on the display screen.

In some embodiments, the apparatus of the present invention may furthercomprise the control device having a trigger input associated with aprocessor, the trigger input may be adapted to be triggered by a userwhich may be configured for operating a computer programme, thereby toenable the processor to send a further signal to an electronic device,the electronic device may be configured to respond to the trigger inputand perform a plurality of outputs.

In some embodiments, the captured image can be processed by the imageprocessor to remove background noise from the image gathered by theimage detection means. This may provide a clearer image for imageprocessing. In some cases, the image processor may remove backgroundnoise to provide a higher quality image for analysis.

In some embodiments, the captured image is processed by the imageprocessor to enhance the image gathered by the image detection means.The enhanced image may provide a higher quality image which may make iteasier for the image processor to recognise the image and process thecaptured image more efficiently. In some embodiments, the captured imagecould be manipulated to enhance the characteristics of the display inthe captured image.

In some embodiments, the image detection means may be suitable for blobdetection, edge detection, corner detection, quadrilateral detection,rectangle or square detection in the image.

In some embodiments, the processed image may be analysed/processed bythe image processor to determine the array of coordinates of the displayon the image based on one or more of the following: contour size,position, intensity, colour, brightness, velocity, shape, outline, edge,angle, light frequency, light polarisation, flickering, refresh rate,contrast and/or size of the display.

Preferably, the captured image may comprise a substantiallyrectangular-shaped or a substantially square-shaped display. Arectangular-shaped or square-shaped display device may comprisesubstantially sharp edges which may mean that the image detection meanscan easily detect the display on the captured image.

As the user may not be directly in front of the display, a rectangularor square display may appear to be a distorted quadrilateral on thecaptured image. The image processing may take into account thisdistortion when calculating a pointing location or position.

The captured image may comprise a display having an outer rectangularshape in order to allow the image processor to determine the array ofimage pixel coordinates of the display consistently with high accuracy.

In some embodiments, the captured rectangular-shaped or thesquare-shaped display may be analysed to determine an array ofcoordinates based on at least one corner or one or two edges of thedisplay. The array of coordinates may be based on more than two or moredifferent corners or edges of the display. Preferably, the imageprocessor may be used to determine the array of coordinates based onfour corners or edges of the rectangular-shaped display.

In some embodiments, the image processor may process the captured imageto determine where the centre pixel coordinates in the image are inrelation to the array of image pixel coordinates of the display.Preferably, the determined display coordinates being pointed at by thecontrol device are represented as X horizontal percentage and Y verticalpercentage.

Optionally, the determined coordinates are represented as a ratio of Xhorizontal percentage and Y vertical percentage.

By using X horizontal percentage and Y vertical percentage ascoordinates, the image processor may be able to process and analyse thecaptured image with the display being at low resolution and/or at highresolution. In other words, the resolution of the display image isirrelevant.

In other embodiments, the determined coordinates may be represented as Xhorizontal and Y vertical pixel coordinates if the resolution of thedisplay is known or X horizontal and Y vertical distance coordinates ifthe size of the display is known.

In some embodiments, the apparatus may further comprise at least onecommunication interface such as a USB interface, HDMI interface, a WIFIinterface or Bluetooth wireless interface, wherein the communicationinterface may be configured to connect the control device to the imagedetection means, the image processor, the electronic device and/or adisplay device.

In some embodiments, the method may further comprise applying an imageartefact, such as a white border or background, which can be applied toshow on the display. The image artefact may also be partiallytransparent. Applying an image artefact to the display screen canoptimise the accuracy of the controller and may also help reduceprocessing requirements. The image artefacts can be applied around theedge, on top of or in the background of the displayed content to makedisplay recognition and processing easier for the processor. The imageartefact may be any change to the display content that enhances theoutline or the shape of the display to make display recognition andprocessing easier for the processor such as during image recognitionprocesses.

In some embodiments, the apparatus may further comprise at least onepower unit in the control device connected to a power source, whichcould be used to provide power to the image detection device and/or theimage processor.

The image detection means may be a digital camera or it may be anyelectro-magnetic radiation image detection means. In some embodiments,the image detection means may be a video recorder, or a mobile phonedevice, or a webcam or any other devices that can be used for detectingan image.

In some embodiments, the display device can be a TV screen, or aprojector screen or a cinematic screen. The TV screen may typically be amodern TV screen. In some embodiments, the TV screen may be a LCD TVscreen or an LED TV screen.

An illuminated display device can be advantageous because it may enablethe image detection means to detect and capture a clear image comprisingthe display. This may then allow the image processor to easily recogniseand process the captured image comprising the display more efficiently.The image process can exclude pixels that do not match a minimumbrightness. The brightness can be used as part of a noise removalprocess during image processing. Optionally, the shape, outline, size,contour level, gamma, angle, light polarisation, brightness, contrast,flickering, refresh rate, colour and/or size of the display may beuseful to enable the image detection means to detect and capture a clearimage that can be easily processed.

In some embodiments, the apparatus may further comprise a storage mediafor storing captured images, the array of coordinates of the displayand/or the coordinates of the pointing alignment/position of the controldevice.

In another aspect of the invention there is provided, a method fordetecting a display using an apparatus according to a previous aspect ofthe present invention, the method comprising the steps of:

-   -   (i) capturing an image comprising the display using an image        detection means;    -   (ii) an image processor receiving a first signal from the image        detection means; and    -   (iii) processing the captured image using the image processor        image to determine an array of image pixel coordinates of the        display which is compared to an image pixel coordinates of the        centre of the captured image;        wherein the image processor further determines a pointing        alignment of the control device in relation to the display on        the basis of the determined image pixel coordinates.

In another aspect of the invention, there is provided a method fordetecting a display using an apparatus described above, the methodcomprising the steps of:

-   -   (i) capturing an image comprising the display using an image        detection means;    -   (ii) applying an artefact to a display content within a display        area of the display, an artefact being applied to a display        content within a display area of the display, wherein the        applied artefact is configured to aid the detection of the        display,    -   (iii) receiving, via an image processor, a first signal from the        image detection means; and    -   (iv) processing the captured image using the image processor        image to determine an array of image pixel coordinates of the        display which is compared to an image pixel coordinates of the        centre of the captured image;        wherein the image processor further determines a pointing        alignment of the control device in relation to the display on        the basis of the determined image pixel coordinates to detect        the display.

Preferably, the method may comprise the step of adding a rectangular ora square border around the inner edge of the display content to assistthe detection of the display area. This may enhance the outline andshape of the display which can be used during the image recognitionprocess to identify the display.

In some embodiments, the method may further comprise the step ofdisplaying the pointing alignment/position of the control device on thedisplay using an electronic device, which may be configured to providean indicator. The electronic device may receive a second signal from theimage processor and may send an output signal to display the pointingposition of the control device on the display.

In some embodiments, determining the array of coordinates of the displaymay be based on contour size, position, intensity, colour, flickering,refresh rate, light frequency, brightness, light polarisation, velocity,shape, outline, contrast, angle or size of the display. In someembodiments, the user may change any one of following based on contoursize, position, intensity, light frequency, light polarisation, colour,flickering, refresh rate, brightness, velocity, shape, contrast, angleor size of the display to optimise the detection of the display. In someembodiments, the method may further comprise the step of applying animage artefact to the display.

In some embodiments, the method may further comprise storing thecaptured image, the array of coordinates of the display device and/orthe coordinates of the pointing alignment/position of the control deviceonto a storage media.

In some embodiments, the method may further comprise the step ofcalculating a changing distance, referred to as a Z axis, from thecontrol device to the display. The Z axis can be calculated as the useroperating the control device moves closer to the display or further awayfrom the display.

In some embodiments, the method may further comprise the step ofcalculating an angle of the pointing alignment/position of the controldevice to the display screen.

In some embodiments, the method of the present invention may furthercomprise validating the captured image comprising the display by afurther image processor, wherein the array of image pixel coordinatesprocessed by the further processor based on one or more of thefollowing: contour size, position, intensity, colour, flickering,refresh rate, light frequency, brightness, velocity, shape, outline,edge, angle, contrast, polarisation, and/or size of the display isvalidated by matching the determined image pixel coordinates determinedby the image processor based on contour size, position, intensity,colour, flickering, refresh rate, light frequency, brightness, velocity,shape, outline, edge, angle, contrast, light polarisation and/or size ofthe display.

In a further aspect of the invention, there is provided a control devicefor detecting a display, the control device comprising:

-   -   an image detection means configured to capture an image        comprising the display,    -   an artefact applied to a display content within a display area;        and    -   an image processor for receiving a first signal from the image        detection means, the image processor is configured to process        the captured image to determine an array of image pixel        coordinates of the display which is compared to an image pixel        coordinates of the centre of the captured image,        wherein the image processor further determines a pointing        alignment of the control device in relation to the display on        the basis of the determined image pixel coordinates.

In another aspect of the invention, there may be provided, a controldevice for detecting a display device, the control device comprising:

-   -   an image detection means configured to capture an image        comprising the display; and    -   an image processor for receiving a first signal from the image        detection means, the image processor is configured to process        the captured image to determine an array of image pixel        coordinates of the display which is compared to a centre image        pixel coordinates in the captured image,        wherein the image processor further determines a pointing        alignment of the control device in relation to the display on        the basis of the determined image pixel coordinates.

In some embodiments, the control device may further comprise a triggerinput associated with a processor, the trigger input can be adapted tobe triggered by a user which may be configured for operating a computerprogramme, thereby to enable the processor to send a further signal tothe electronic device, the electronic device may be configured torespond to the trigger input and perform a plurality of outputs. In someembodiments, the control device may comprise the electronic device.

In some embodiments, the control device may further comprise a vibrationmotor, speaker or a solenoid interconnected to or in association withthe trigger input and/or additional inputs, and/or triggered by theelectronic device, wherein the vibration motor, solenoid or speaker maybe configured to provide feedback upon the trigger sensor beingtriggered by a user. As an example, the vibration motor can be used toprovide haptic feedback. As another example, the solenoid can provide arecoil effect.

In some embodiments, the control device may further comprise anadditional button and/or a scrolling wheel or an additional control orany other input controls. These additional buttons and scrolling wheelmay provide a plurality of different functionality for the usercontrolling the control device. For example, the scrolling wheel may beuseful to activate to scroll the content that may be displayed on thedisplay device. The additional button may be used to control the imagedetection means. An additional control may be used to control the imageartefacts added to the display content, for example changing one or moreof the following but not limited to the thickness, size, shapes orcolour of the border.

In another aspect of the invention, there is provided a kit fordetecting a display according to any of the previous aspect of theinvention, the kit comprising:

-   -   a image detection means for capturing an image; and    -   an image processor configured to process the captured image.

In some embodiments, the display can be a television, a laptop screen, acomputer screen, a projector screen, a cinematic screen, mobile phonescreen, tablet, or on any other electronic screens, wherein the displayis a modem technology display screen such as an LCD or LED screen.

In some embodiments, a border could be added around the display contentusing a physical modification such as an illuminated neon strip. Thisphysical method would still utilise the shape detection and brightnessdetection of the software method but would not require the ability toadapt the display content. This would be useful for example on an oldarcade gaming machine where it is not possible to modify the displaycontent.

The invention will now be further and more particularly described, byway of example only, and with reference to the accompanying drawings, inwhich:

FIG. 1 shows an apparatus according to an aspect of the presentinvention,

FIG. 2a , provides a side view of a control device according to FIG. 1,

FIG. 2b , shows a cross sectional view of the control device accordingto FIG. 2 a,

FIG. 3a to 3c shows a captured image comprising a display,

FIGS. 3d and 3e shows an array of plotted coordinates of the display,

FIG. 4 provides an illustration of a detecting a display at an angleaccording to FIGS. 1a, 1b , 2, 3 a to 3 e,

FIG. 5a to 5h show the display device at different distance and/orangles according to FIGS. 1 to 4;

FIG. 6 shows an upward and downward direction of the control device;

FIG. 7a shows a display with no border and FIGS. 7b to 7f show a displaywith an image artefact added to the outline/edges/corner of a displaycontent; and

FIG. 8a to 8b shows a representation of a light polarisation filter.

As used herein, unless otherwise stated, the image processor may bedefined as a processor which is processing the captured images and/or itmay include processing user inputs such as buttons and triggers and/orsending communication signals to an electronic device comprising datarelevant to the process e.g. captured images or calculated coordinates,or it may provide any other processing means required on the device.

Referring to FIGS. 1, 2 a and 2 b, there is provided an apparatus 10 fordetecting a display device 14 such as a television. The apparatus 10comprise a control device 12, an image detection means 16, which may beconfigured to capture an image comprising the display 13 and an imageprocessor 18 for receiving a first signal from the image detection means16. The image detection means 16 may be a digital camera or a webcamwhich can be used to take images or videos which comprises the displaydevice 14. Alternatively, the image detection means 16 may be used todetect a live image. The image processor 18 may receive a first signalof the captured image from the image detection means 16 such as acamera. The image captured by the camera may then be processed by theimage processor 18. The image processor may be in a form of a CPU/masterunit on an electronic device 20 for example a computer device.Alternatively, the image processor may be built within the controldevice and/or the camera.

Referring to FIGS. 2a and 2b , there is shown an example of a controldevice 12. The control device may be a remote controller or it may be agaming accessory device, such as a gun accessory as illustrated ormodified game controller. In other examples, not illustrated in thesefigures it may be a mobile phone device or a computer. The controldevice may be used to calculate and provide feedback where the pointingalignment/position is at the display 13, as shown in FIG. 1. The controldevice 12 may operate at a distance from the display or the controldevice may operate at an angle to detect a distorted shape comprisingthe display, as shown in FIG. 4.

The control device may have an aperture 15 in which the image detectionsmeans 16 may be positioned in such a way to enable the lens of the imagedetection means 16 to capture an image. In some embodiments, the controldevice may have more than one aperture. In some embodiments, a pluralityof image detection means may be provided. The camera 16 or any otherimage detection means may be built into the controller 12.Alternatively, the camera 16 may be mounted or attached on top of asurface of the control device 12. As the camera may be mounted onto thecontrol device, the centre of the camera would point in line with auser's line of sight. Where the camera 16 points to may be equal to thepointing direction 19 of the control device 12. The camera then recordsan image comprising the display as shown in FIGS. 3a and 3b . The cameramay take a 1D, 2D or 3D image.

The camera may take an image at any resolution e.g. at high or at lowresolution. The image detection means may be connected to an electronicdevice 20 e.g. a computer using a communication interface such asthrough a USB cable or a HDMI cable 22. Furthermore, the camera or thecontrol device may be connected to the electronic device 20 viaBluetooth or any other wireless communication means. For example, thewireless communication may utilise a wireless internet connection suchas WIFI or 1G, 2G, 3G, 4G, 5G and beyond.

As shown in FIGS. 1, 2 a and 2 b, there is also shown a primary selectcontrol or a trigger input 21. The trigger input is interconnected witha trigger sensor/button 27 which is interconnected to the imageprocessor 18 via the cable 22. The trigger input 21 can be associatedwith a processor 18. Furthermore, the trigger input 21 can be adapted tobe triggered by a user which may be configured for operating a computerprogramme, thereby to enable the processor 18 to send a further signalto the electronic device 20. The electronic device may then beconfigured to respond to the trigger input and perform a plurality ofoutputs.

In other embodiments not shown in the accompanying drawings, theplurality of outputs may be configured to perform a variety offunctionality including, but is not limited to, being triggered by theuser to perform one or more of the following; a mouse click/movement,keyboard button press or joystick movement/button press.

The controller can have further additional controls 23 that may beoperable by a user. The additional control 23 may be an operable controlsuch a secondary trigger input, buttons or scroll wheels in which aparticular functionality can be assigned/mapped towards providing theuser benefit in the computer programme being operated. As an example,the secondary trigger input 23 may be associated with a mappedfunctionality that is similar to the function of a right-mouse click,when operable by the user. The controller may be a hand-held device. Thecontroller can have a suitable surface 25 to allow a user to hold/gripthe controller. As illustrated in FIGS. 1 and 2, the controller may alsocomprise at least one connection port 29 which can be connected to a USBport or a HDMI port.

In addition, the control device may have a visual front indicator 50(front view) and a visual back indicator 52 which may be configured toallow a user to line up where the control device is pointing e.g. a gunsight.

As an additional feature the control device 12 could contain a displayscreen such as a LCD screen which may function as a line of sightindicator or gunsight. This display may be connected to the imageprocessor in the control device. The image processor would output azoomed in image from the control device camera possibly with anover-laid cross hair onto this additional screen. The image may bezoomed in at the centre of the image which is where the control deviceis pointing. This may be an additional way to indicate where the controldevice is pointing at to the user. The LCD screen may be used foroutputting configuration options meaning the user can configure orcontrol the device for feedback. The LCD screen may be a touchscreen toallow the user to add further inputs.

The image processor 18 may run a software algorithm to process thecaptured image 17, as shown in FIG. 3a , to determine an array of imagepixel coordinates 34 of the corners of the display 13 in relation to acentre image pixel coordinates 36 or 28 in the captured image 17, asshown in FIGS. 3a, 3b, 3c, 3d and 3e . In FIGS. 3a and 3b , the centreimage pixel 36 or 28 coordinates represents the pointingalignment/position or line of sight of the controller in relation to thedisplay screen 13. The centre image pixel of the captured image isindicated by a crosshair 28.

The image processor 18 may be interconnected with the control device 12.In some embodiments, the image processor can send and/or receive aplurality of input or output signals to communicate with the controldevice. The image processor can be configured to send a second signal toan electronic device 20. The electronic device 20 may then provide anoutput signal to show the pointing alignment/position of the controldevice on the display via an indicator, such as a mouse cursor.Furthermore, the image processor may provide an output signal configuredto enable a mouse movement/joy stick movement that corresponds to thepointing alignment/position of the control device on the display.Additionally, the trigger and additional buttons on the gun accessorymay provide mouse clicks and/or joy stick button clicks.

The image processor may provide additional processing capabilities suchas interpreting the control device controls for example the triggerinput.

The electronic device such as a computer device 20 may be an externalcomputer hardware that can be connected to the image processor, thecontrol device, image detections means and/or the display device througha USB or a HDMI connection means 22. The electronic device can comprisea connection port such as a USB or a HDMI port 51.

In some embodiments, the computer device may be built within the controldevice. The control device could connect directly to a display andoutput content. The computer device 20 may be set up to receive a secondsignal from the image processor 18, which may be raw image data, so thatthe computer device can provide an output signal to display the cursorthat corresponds to where the control device is pointing to at thedisplay. In some embodiments, the output signal may be configured todisplay a constantly moving mouse cursor in line with where the camerais pointed on the television or only to move the cursor when then thetrigger on the control device is clicked.

In some embodiments the display device could also include the computerdevice, for example, the display device may be a smart television.

Referring to FIGS. 3a, 3b and by way of example only, pointing thecontrol device at exactly in the middle of the television would be equalto X horizontal percentage=50% and Y vertical percentage=50%. By usingpercentages it does not matter what the resolution or the aspect ratioof the display is. In the example illustrated in FIGS. 3a and 3b , theimage resolution is 640 horizontal X pixel x 360 vertical Y pixel. Thecentre image pixel, as represented by a crosshair 28, has a pixelcoordinates of 320 X horizontal; 180 Y vertical coordinates, whichcorresponds to the pointing alignment/position of the control device onthe display.

Typically, the image processor 18 contains a software image recognitionalgorithm, which is used detect a display 13 having four edges and/orfour corners. The display device 14 has a display screen 13. The displayscreen 13 may be a substantially rectangle-shaped or square-shapeddisplay screen. The display would typically be brighter than the generalbackground around and behind the display.

The edges or the corners 30 of the rectangle-shaped or square-shaped 32display screen 13 may have an angle of between 45° to 120°. In someembodiments, the edge/corner of the rectangle-shaped or square-shapeddisplay screen may have an angle that can be more than 45, or 50, 60,70, 80, 90, 100, 110° or 120°. In some embodiments, the edge/corner ofthe rectangle or square shaped display screen may have an angle that maybe less than 90, 80, 70, 60, 50, 40, 30, 20° or 10°. Preferably, theedge/corner 30 of the rectangle- or square-shaped 32 display screen 13has an angle that is substantially 90°. This is advantageous because theedge/corner of an object having an angle of about 90° can be easilydetected and processed by the image processor 18. Objects having sharpedges/corners around 90° can also be easily processed by the imageprocessor 18 to determine the array of coordinates of the display.However when the control device 12 is at an angle to the display device14 the corner angles will not appear to be 90 degrees, so the imageprocessor should be configured to process a distorted square orrectangle and thus may use quadrilateral detection to identify arectangular or square shaped display.

The image processor processes the captured image to determine an arrayof coordinates 34 that corresponds to the edges or corners 30 of the TVscreen as shown in FIG. 3c . For example, an array of X, Y coordinatesmay represent the four corners or edges of the rectangular TV screen.The image processor can further collect and process an n^(th) number ofcoordinates that corresponds to the outer edges of the TV screen. Insome embodiments, the coordinates may be X, Y, Z coordinates. In someembodiments, the coordinates are shown as X horizontal percentage and Yvertical percentage. In some embodiments, the coordinates may be 1D, 2Dor 3D coordinates.

Referring to FIGS. 3c and 3d , the X horizontal percentage and Yvertical percentage coordinates of the centre image pixel 36 iscalculated using a mathematical algorithm in relation to the array ofthe coordinates 34 corresponding to the edges or corners 34 of thedisplay screen 13.

In the example as shown in FIG. 3e , the pointing alignment/position ofthe control device is calculated at approximately 60% along the x axis38 and approximately 80% down the y axis 40. The image processor maycalculate the percentage coordinates using a CPU/master unit on acomputer or it can be calculated using a local processor built withinthe control device.

Referring to FIG. 4, the apparatus of the present invention can be usedto detect the display screen at any angle. The captured image comprisingthe display screen may be distorted due to the viewing angle of thedisplay, for example keystoning effect. This may result in one side ofthe rectangle of the display device being slightly shorter 42 on thecamera image than another side of the rectangle 44 when viewing at anangle. As the image processor is able to determine the X, Y image pixelcoordinates of the display, the image pixel coordinates of the centre ofthe image and is also able to know that the display screen is a fixedrectangle or square, the image processor may be used to run an algorithmto compute, with accuracy, the pointingalignment/position/angle/distance of the control device 12 in relationto the display at an angle.

To optimise the accuracy of determining the pointing alignment/positionof the control device at the display screen, and to reduce processingrequirements of the captured image, an image artefact can be added ontop of the displayed content to make display recognition easier for theimage processor, for example a white border/background around the edgeof the display.

The image processor may process the image using the following algorithm:

Colour and brightness filters—Decreasing the brightness reduces thebackground noise and the brighter objects such as the display imagecontinue to be well defined on the image. Removing the colour and makingthe image greyscale simplifies the algorithm.

The algorithm may filter to only keep a certain colour. For example if ablue border was used to enhance the outline of the display then thealgorithm may use a filter to remove all captured light that is notblue.

Blob detection—The remaining strongly defined objects are recorded intoan object array. These are solid patches of brightness across areasonable size area. The algorithm may then remove all the smallobjects that don't match a minimum size.

Edge detection—To detect edges where the image strongly changes. Thismay be applied to all the blobs to turn them into Polygons.

Corner detection—The edges/corners in each blob are analysed and if theymeet in a corner these corners are identified as being joined edges andare therefore corners.

Quadrilateral detection—Using the corners/edges of the display device tolook for objects that have 4 corners/edges.

Square or rectangle detection—Compare the ratios and angles of the 4sides/edges to see if they are within the limits of a rectangle orsquare.

Alternatively, the algorithm on the image processor may include afunctionality to detect the outer frame of the display device andutilise the outer frame of the display device for image recognition tocalculate the edges of the display.

Referring to FIG. 5a to 5c , there is shown a substantiallyrectangular-shaped TV screen on the image. FIG. 5b shows how the shapeof the TV on the image changes as the controller move further away, i.e.moves further away from the display device while FIG. 5c illustrates atypical image detected by the image detections means as the controllermoves to closer to the display device.

By determining and knowing the array of coordinates of the displayscreen, i.e. the coordinates of the outline shape of the display screen,as described above in FIGS. 3a to 3e , and calculating an overall sizeof the display screen, for example the average of the 2 diagonal cornerto corner points of the display, a relative z axis value (distance ofthe control device from the display) can be calculated as the controldevice moves further away or moves closer to the display.

This z axis relative value can also be provided to the electronic deviceas an additional input that can be utilized. An example would be ashooting computer game where the user would be moving further away orcloser to a target and the electronic device, which could represent thatinside the virtual world of a shooting computer game. If a z-axisdistance for a particular point is known, or the size of the display isknown, then the z axis relative value can be used to give a z axisdistance.

FIGS. 5d to 5f , and FIGS. 5g to 5h show examples of the display screenbeing viewed from different angles. As the coordinates of the cornersare known using known mathematical calculations, it is then possible tocalculate at what angle the display screen is being pointed at from bythe control device. In the examples as illustrated in FIGS. 5d to 5f ,an angle may be represented as 2 angles which can then be provided tothe electronic device as additional inputs that can be utilised. Theangle may be an x angle, which may represent the right and left(horizontal) pointing angle of the control device pointing at thedisplay screen, and a y angle, which may represent an up and down(vertical) pointing angle of the control device. The angle at 0 degreesmay be configured to mean straight on and −90 degrees to +90 degrees mayrepresent 180 degrees of a viewing angle.

On the x angle, −90 degrees may be the user operating the control devicelooking at the display completely sideways on from the left and +90degrees may be the user operating the control device is lookingcompletely at the display screen sideways from the right. On the y angle−90 degrees would mean the user could be pointing the control devicefrom the bottom and +90 degrees may be the user pointing the controldevice looking from the top. These inputs could be used by theelectronic device in for example a shooting computer game where the useris looking through a window. By changing their pointing angle, which isprovided to the computer game, it could represent the user lookingthrough the window at a different angle and this could be shown in thecomputer game adding an additional user experience to the user. It ispossible to generate a three dimensional effect to the user if thedisplay content is correctly adjusted based on the known position andorientation of the control device, assuming that the view point of theuser relative to the display is approximately the same as that of thecontrol device.

The control device may be used to detect a display at any angle of atbetween −90° to +90°. In some embodiments, the angle may be more than−90°, −45, 0° or +45°. In some embodiments, the angle may be less than+90°, +45°, 0° or −45°.

Referring to FIG. 6, there is shown a control device 12 such as a gaminggun, which may comprise a gyro sensor (not shown) to provide additionalinput to the electronic device. This input may provide usable input whenthe camera is not pointing at the display screen 13 and the controldevice 12 is effectively blind. An example might be a cowboy shootingcomputer game. Where in the game the user puts their virtual gun in theholster for a cowboy duel. In some embodiments, the user may point thecontrol device 12 in a downward direction 62, i.e. towards the floor,and then draw and shoot their gun by pointing it at the display screen13, which can all be simulated in the virtual game for user experience.In some embodiments, the user may point the control device 12 in anupward direction 64 to simulate either cocking the gun or reloading thegun which can be simulated in the virtual game for user experience. Thegyro sensor may be connected to the image processor and its output maybe provided to the electronic device. The control device can havemultiple gyro sensors to provide usable inputs across multiple axis.

The gyrosensors can detect if the control device has been rotated forexample, upside down and can therefore communicate to the imageprocessor that it needs to vertically flip the image to continue toprovide accurate pointing coordinates.

Referring to FIG. 7a , there is provided a television 100 containing atelevision bezel 102 surrounding the television display image/picture104. Referring to FIG. 7b and FIG. 7c , a border 106 is added around theedges or corners of the display content. As shown in FIG. 7b , theoriginal display content 104 may be resized in order to fit withinborder 106. The border may be detected by the detection means accordingto the present invention. Referring to FIG. 7c , the border may beoverlayed on top of the original content.

Border

Referring to FIGS. 7a to 7c , an artefact for example, a background orborder may be added around the edge or on top of the edge of the displaycontent that can be detected by the imaging detection means and detectedin the software using image recognition, which could be shape, colour,brightness or a pattern. In the simplest example but by no means limitedto this example, a border can be a solid colour with a set number ofpixels width around the edge of the display content. With additionalsoftware calculations the image recognition can also work on a partialborder outline. This may be to make the border less visible to the useror because part of the border is outside of the imaging apparatus'sfield of view. The border makes it easier for the software to identifythe rectangle, square and/or quadrilateral edge of the display area andtherefore the corners of the display area. This means it can determinewhere the control device is pointing at the display with greateraccuracy, better reliability and faster. Referring to FIGS. 7d to 7f ,the border could also be made more complex, so that even if the field ofview of the image detection means does not include the whole border, theportion of the border that it is visible will still contain enoughinformation for the image processor to calculate which portion of thedisplay it can see.

FIG. 7d shows an example of an octagonal border that is substantiallyrectangular in shape. This can be detected using similar imageprocessing algorithms to detecting a quadrilateral, as described above,

FIG. 7e shows an example of a captured image where the whole of thedisplay is not visible. The display is surrounded by the same border asthat shown in FIG. 7d . The additional corners at the midpoint of thedisplay edge allow the pointing alignment to be calculated withoutseeing the complete display. As an example, the pixel coordinates 110that would be identified in such a situation are highlighted in FIG. 7f.

In some embodiments, the border could be patterned and the imageprocessing algorithm could identify the border by image recognition ofthe pattern. This approach may remove noise and could be used inconjunction with the shape and brightness recognition methods describedabove.

Polarisation

The performance of the device may be improved by using an imagingapparatus capable of detecting light polarisation. LCD televisions andmost other modem displays emit light that is polarised in one direction.Referring to FIGS. 8a and 8b , by manually calculating or byautomatically detecting the polarisation direction of the display screenlight the imaging apparatus or image processor can exclude other lightthat is not polarised or not polarised in the same direction as thedisplay. This will reduce unwanted light which is providing noise to theimage recognition and therefore improve performance. This technique maywork especially well when combined with the adding of the rectangleborder around the edge of the display screen. When the rectangle borderis detected by the image recognition software the polarisation can berecorded. Then on subsequent image frames light that is not polarised ina similar direction or is not polarised at all can be excluded whichshould emphasise the border even more. If the imaging apparatus isrotated then the polarisation direction might no longer be accurate. Ifthe rectangular border edge cannot be identified on an image then thepolarisation exclusion will need to be removed so the rectangle borderedge can be detected. However the new polarisation direction can then beidentified and the exclusion used again on subsequent frames.

Referring to FIG. 8a , there is shown a television 100, a televisionbezel 102 surrounding the television display image/picture 104. In anexample, the recorded image can be readied for processing using imagerecognition algorithms. Referring to FIG. 8b , a polarisation imagefilter 108 has been put in place in order to only keep the light frompixels where dominant polarisation direction value matches thetelevision display light polarisation direction. This may provide astrong filter to remove unwanted light which is providing noise inrelation to the image recognition.

FIG. 8b shows how this polarisation based filtering can make the imageprocessing significantly easier, efficient, more accurate and reliable.This process does not necessarily involve changing the lightpolarisation of the display, but may merely involve exploiting theexisting light polarisation information.

Changing the light polarisation of the display or other light sources inthe vicinity of the display to for improved functionality using theabove described process is also possible Alternatively, a configurationwhere display light is not polarised, but some of the background lightis polarised, and the light polarisation of the background light isutilised to exclude it from the imaging apparatus or image processing ispossible. Alternatively, it could be that the display light is polarisedin one direction and the background light is polarised in a differentdirection and this information is utilised as described above.

In some embodiments, the image detection means may comprise two camerasadjacent to each other, one camera with a polarisation filter and theother camera without a polarisation filter, and detecting the displaymay comprise comparing the relative brightness of images detected by thetwo cameras to detect the display, which will be brighter in one imagedue to the filtering of polarised light.

Alternatively, instead of applying a polarisation based filter, it wouldbe possible to aid detection of the display by recording thepolarisation of light detected by the image detection means at eachpixel.

Close Mode

If the detection device is being used close to the display device theimaging apparatus may be able to see the whole display screen whenpointing near the middle of the display but when pointing at the edgesit may not be able to see the whole display screen as some of it will beoutside of the imaging device's field of view. This means the softwaremay only be able to identify a partial rectangle outline of the displayedge with the rest of the rectangle edge of the display screen outsideof the image. The image recognition software can identify that therectangle is only a partial match and that the rest of the rectangle isoutside the edge of the captured image. The software can estimate wherethe other rectangle corners might be by assuming the rectangle is thesame size as it was when it last captured a full rectangle match. Theimaging device has to be able to identify at least one corner of thedisplay screen edge for this to work. This feature would enable thedevice to work closer to the display screen than it otherwise would.

Example 1—Hardware GunMouse

The control device as referred to herein in these examples (below) as aGunMouse. The camera may be mounted onto the GunMouse or it may be builtinto the front of the GunMouse. The GunMouse may be connected to orattached to an image processor for example, a small computer or a singleboard computer such as a Raspberry Pi, Arduino or a custom-builtcomputer for this purpose.

This small computer located in the GunMouse may process the images andwork out where the camera is pointing at on the display screen forexample a television screen. The small computer then processes the imageto calculate the array of image pixel coordinates in the imagerepresenting the four corners of the display. By calculating the arrayof image pixel coordinates, the image processor may then be able tocheck and compare those coordinates against the centre image pixelcoordinates of the image so that it can determine where on the displaythe control device is pointing at, and translate these into mousemovements which can be sent to the electronic device such as a maincomputer. The main computer may then provide an output signal to showwhere the pointing alignment/position of the GunMouse is on the displayscreen via an indicator such as a standard mouse cursor to display thepointing alignment/position of the GunMouse onto the display screen. Thepointing alignment/position of the GunMouse on the display screen can beequal to the centre image pixel of the captured image. Usually, the maincomputer and the GunMouse are connected through a standard USBconnection.

The GunMouse communicates to the main computer using standard mousecommunication interface such as the HID standard. The version describedin this example does not require any drivers different to a normal mouseand can easily be used for anything that a normal mouse can be used for.The GunMouse receives its power through the USB connection to the maincomputer.

Example 2—Software GunMouse

The difference with this version over Example 1 is that the camera feedis fed into the main computer where it is processed by custom software/adriver on the main computer. When the software has worked out where themouse should point to on the display screen, it interfaces with theoperating system of the main computer to create a virtual mouse whichcan then move and be used in all applications just like a normal mouse.

Alternatively, the main computer could feed the calculated target mousecoordinates back into the GunMouse so it can pass them back to the maincomputer as a standard mouse signal similar to Example 1.

The trigger and any additional buttons/controls/interactions are sentvia a small serial GPIO board located in the GunMouse, which may beconnected to the main computer. Both the camera and the GPIO board canbe connected to the main computer most likely by a single USBconnection. The custom software/driver on the main computer may processthe trigger and button clicks/control operations and perform therelevant virtual mouse actions, or feed these back into the GunMouse tooutput back into the main computer as normal mouse and/or keyboardand/or joystick signals.

The GunMouse can communicate to the main computer if the GunMouse isactually slightly pointing away from the screen and in which direction.This information may be utilised for additional functionality forexample, when in combination with a trigger event for example, reloadinga gun in a computer game. This is known as off-screen reload.

Example 3—Mouse Coordinates

The gun mouse primarily deals with horizontal and vertical percentageswhen calculating coordinates as it does not know the resolution oraspect ratio of the screen. The HID mouse standard supports theseabsolute percentages. For example (0,0) represents the top left of thedisplay and (65/535,65535) represents the bottom right. If the GunMouseis pointed around the middle of the screen, which gives x horizontalpercentage of 50% and a y vertical percentage of 50%, the centre of thedisplay screen would be communicated as (32767,32767) absolutecoordinates to the main computer. The software having worked out thehorizontal and vertical percentages may pass that in as a percentage of65535. The main computer operating system may then use the providedabsolute mouse coordinates to move the mouse cursor to the correctlocation on the screen, which in this example is the centre of thedisplay.

However some devices/computers may not properly support absolute mousecoordinates. The GunMouse can be configured to use relative mousecoordinates but it may need to know the horizontal and vertical totalpixel count. On a software GunMouse as described in Example 2 above,this can be detected on the main computer so can be easily utilised formoving the mouse.

On the hardware mouse as described in Example 1, this may have to beinputted some way. For example controls on the GunMouse, a connectioncable between the main computer and the GunMouse computer to pass thesetting, or an SD card slot on the GunMouse where a user can change asetting on the SD card.

Example 4—GunMouse for Modern Gaming

It may be possible to use the GunMouse in combination with another gamecontroller in a computer game. For instance, it may possible for a userto move a 3D first person perspective computer player using a joystickwith one hand and shoot targets using a GunMouse in the other hand. TheGunMouse can be compatible with any console games and PC games. Thecontrol device may be configured to interact with other accessory itemsfor example a gaming accessory such as a pedal.

Example 5—Playing Classic Lightgun Computer Games

The GunMouse can be connected to a main computer where it is providingmouse input. The main computer can run emulation software that is ableto run classic or retro computer games. It is a standard feature ofclassic computer game emulators that lightgun games can be controlled bya mouse. This is because classic lightgun technology does not work onmodem display devices and this is an alternative way to play but not asmuch fun for the user.

The GunMouse can be used with this software as it is interpreted as amouse. The user interaction then becomes exactly the same as a classiclightgun in that it does not require additional external hardware to beused along with the gun and it does not require calibration. This ismore fun for the user and a similar user experience to the original userexperience.

The GunMouse when configured to be a mouse can be configured toconstantly move the mouse cursor in line with where it is pointing atthe display or only move the cursor when the trigger is actioned. Whenmoving when the trigger is actioned the mouse cursor would be movedfirst when the trigger is actioned and then the mouse click event wouldbe applied. These 2 modes are useful because some classic light gungames constantly showed a moving cursor that represented where the gunwas pointing and some classic light games only interacted with the gamewhen the trigger was actioned. On the latter if you had a cursor showingwhere the gun is pointing this would be different to the original userexperience however the option is there for the user if they want to playin that mode.

It may not be a requirement to show a cursor or a crosshair on thedisplay screen as the computer may still interact with the GunMousewithout it. This is a user preference.

Example 6—GunMouse in Presentations, Training and Education

Whilst the GunMouse may be used for playing computer games, the GunMousemay give opportunity for lots of new ways to interact with a computer.For example a user could utilise it for a computer presentation toincrease interest. Optionally, the user may use the GunMouse to shootdown targets representing company business targets that had beenachieved.

The GunMouse could be used for training and education as it may providenew ways to simulate real world interactions virtually.

It will be appreciated that the number of image detection means and/orimage processors may vary substantially. The number of additionalbuttons on the control device may also vary substantially. The array ofcoordinates processed by the image processor can vary substantially.

Various further aspects and embodiments of the present invention will beapparent to those skilled in the art in view of the present disclosure.

All documents mentioned in this specification are incorporated herein byreference in their entirety.

The terms “centre pixel coordinate” and “image pixel coordinates of thecentre of the captured image” as defined herein are to be taken to meanany pixel coordinates on a given image which are not towards theextremes of the display on which that image is being displayed, andshould not be construed to mean pixel coordinates at exactly the centreof the image.

The term “display” as defined herein is to be taken to mean the area ofa display device on which content is displayed, and does not include,for example, the physical border surrounding a television screen.

The term “border” as defined herein is to be taken to mean any artefactthat is applied to a display of a display device and which completelysurrounds at least a portion of the display of that display device.Similarly, the term “partial border” as defined herein is to be taken tomean any such artefact which partially surrounds at least a portion ofthe display of a display device. Such borders and partial borders neednot necessarily be continuous, and may for example comprise dotted linesapplied to surround or partially surround a portion of a display.

“and/or” where used herein is to be taken as specific disclosure of eachof the two specified features or components with or without the other.For example “A and/or B” is to be taken as specific disclosure of eachof (i) A, (ii) B and (iii) A and B, just as if each is set outindividually herein.

Unless context dictates otherwise, the descriptions and definitions ofthe features set out above are not limited to any particular aspect orembodiment of the invention and apply equally to all aspects andembodiments which are described.

The term “comprising” encompasses “including” as well as “consisting”e.g. a composition “comprising” X may consist exclusively of X or mayinclude something additional e.g. X+Y.

The word “substantially” does not exclude “completely” e.g. acomposition which is “substantially free” from Y may be completely freefrom Y. Where necessary, the word “substantially” may be omitted fromthe definition of the invention.

The term “about” in relation to a numerical value x is optional andmeans, for example, x±10%.

It will further be appreciated by those skilled in the art that althoughthe invention has been described by way of example with reference toseveral embodiments. It is not limited to the disclosed embodiments andthat alternative embodiments could be constructed without departing fromthe scope of the invention as defined in the appended claims.

1-20. (canceled)
 21. A method for detecting a display, comprising:receiving an image signal from an image detection means; detecting, byan image processor, that the received image signal comprises an image ofat least a portion of a display; determining, by the image processor, anarray of pixel coordinates of the received image signal that define atleast a part of an outline of the display; and calculating, based on thearray of pixel coordinates and at least one centre pixel of the receivedimage signal, an orientation of the image detection means in relation tothe display.
 22. The method of claim 21, method further comprising, inresponse to calculating the orientation of the image detection means,communicating a pointing alignment of the image detection means relativeto the display to a processor in communication with the display.
 23. Themethod of claim 22, wherein the processor in communication with thedisplay is configured to position an indicator on a display content areaof the display based on the received pointing alignment.
 24. The methodof claim 21, further comprising adjusting the content of the displaycontent area of the display for improved recognition by the imageprocessor.
 25. The method of claim 24, wherein adjusting the contentcomprises applying an artefact to the display content area, and whereinthe artefact is a border or a partial border.
 26. The method of claim24, wherein adjusting the content comprises the step of adding abackground and/or foreground to at least a portion of the displaycontent area and wherein the background and/or foreground is partiallytransparent.
 27. The method of claim 24, wherein adjusting the contentcomprises applying a colour and/or brightness adjustment to the whole ofthe display content area.
 28. The method of claim 24, wherein adjustingthe content comprises applying a colour and/or brightness adjustment toonly a part of the display content area.
 29. The method of claim 28,wherein the colour and brightness adjustments are applied to portions ofthe display based on the values of one or more pixels in those portions.30. The method of claim 21, wherein the image detection means isconfigured to detect the polarisation of light impinging on the imagedetection means, and wherein the image processor is configured to detectthe display based at least in part on the detected polarisation.
 31. Themethod of claim 21, wherein the step of determining the array of pixelcoordinates is based at least in part on an expected shape of theoutline profile of the display.
 32. The method of claim 31, wherein theshape of the expected outline profile is a square or rectangular and/orquadrilateral profile.
 33. The method of claim 31, wherein determiningthe array of pixel coordinates is based at least in part on thebrightness of the display relative to a background.
 34. The method ofclaim 21, further comprising calculating a distance between the imagecapture means and the display based on the array of pixel coordinates,detecting a change in the distance between the image capture means andthe display, and adjusting the content of the display based on thechange in distance.
 35. The method of claim 21, further comprisingcalculating a relative angle between the image capture means and thedisplay based on the array of pixel coordinates, detecting a change inthe relative angle between the image capture means and the display, andadjusting the content of the display based on the change in angle. 36.The method of claim 21, wherein the step of detecting that the receivedimage signal comprises an image of at least a portion of the displaycomprises analysing the image based on one or more parameters detected,wherein the parameters include at least one of: contour size, position,intensity, colour, flickering, refresh rate, light frequency,brightness, velocity, shape, outline, edge, angle, contrast, lightpolarisation, and/or size of the display.
 37. An apparatus for detectinga display configured to carry out the method of claim 21, the apparatuscomprising: an image detection means; and an image processor configuredto be in wired or wireless communication with a display.
 38. Theapparatus of claim 37, wherein the apparatus further comprises: apointing device configured to be in a fixed orientation with respect tothe image detection means; and/or a trigger input configured to be inwired or wireless communication.
 39. The apparatus of claim 37, whereinthe control device is configured to communicate with the display toprovide the functionality of a lightgun.
 40. A computer-readable mediumcomprising instructions which, when executed by a processor, cause theprocessor to carry out the method of claim 21.